Image pickup apparatus with horizontal line interpolation function having three image pickup units shifted in vertical phase from one another

ABSTRACT

In an image pickup apparatus with a horizontal line interpolation function having a plurality of solid state image pickup elements, three chrominance signals obtained from the plurality of solid state image pickup elements are shifted in vertical phase from one another, whereby when a horizontal interpolation line is synthesized, in the case where the interpolated video signal is viewed as a whole of three chrominance signals (e.g. when a luminance signal synthesized from G, R, and B signals by matrix operation is considered), deterioration in vertical frequency response characteristic can be reduced, and deterioration in vertical sharpness of an image can be reduced. Further, by combining a frame signal in a pseudo fashion, a high-quality frame still image can be formed.

TECHNICAL FIELD

The present invention relates to an electronic zoom function circuit, amotion correction circuit for implementing motion correction, and afunction of obtaining frame image signals of high image quality in animage pickup apparatus such as a video camera, and more specifically,relates to an image pickup apparatus with a horizontal lineinterpolation function having a horizontal line interpolation functionin which an electronic horizontal line interpolation process isimplemented by arithmetically processing a video signal, and having afunction in which a pseudo frame signal is generated by arithmeticallyprocessing a video signal.

BACKGROUND ART

In recent years, in image pickup apparatus such as video cameras, theapparatus have been advanced with reduction in size and weight,high-multiplification zooming, and moreover multi-function, so thatproduct development is undergoing in which the optical zoom andelectronic zoom functions are interlocked. Also, as users have spreadover from children to the old as well as conventional enthusiasts,screen blurs due to hand blurs are generated, and there have beenmarketed image pickup apparatus provided with a motion correctioncircuit using electronic zoom function.

A conventional motion corrector device using the electronic zoomfunction is shown in, for example, TV society technical report Vol. 11,No. 3 (May. 1987).

The conventional motion correction circuit having an electronic zoomfunction is described below.

An input signal is converted into a digital signal by an A/D converterand fed to a motion vector detector circuit and a memory circuit, and inthe motion vector detector circuit video signals of two fields arecompared to detect a motion vector, and in a memory control circuitusing the motion vector to obtain a stub signal corrected for hand blursfrom the memory circuit, and a memory output signal is formed into anormal video signal by an interpolation circuit which is controlled byan interpolation control circuit and is converted into an analog signalby a D/A converter circuit.

In this way, a motion vector is detected from two-field video signals,and hand blurs are corrected by the interpolation function.

As another example of the conventional image pickup apparatus having ahorizontal line interpolation function, there is listed, e.g., "An imagepickup apparatus" of U.S. Pat. No. 4,951,125 (registered on Aug. 21,1990), which was filed by the present applicant and registered.

The conventional motion correction circuit having an electronic zoomfunction is described below.

The above conventional example comprises: a solid state image pickupelement; an image pickup element drive circuit for controlling thetransfer and halt of vertical transfer (scanning) of the solid stateimage pickup element with a control signal (denoted by CTL1); aswitching unit for apportioning an output signal SO of the solid stateimage pickup element according to a control signal (denoted by CTL2) toa first line memory M1 through an nth line memory Mn (n≧3, where n is aninteger); a selector for selecting output signals of m lines out of theline memories M1 through Mn (2≦m<n, where m is an integer) according toa control signal (denoted by CTL3); one or a plurality of multipliersfor multiplying output signals S1 through Sm of the selector withweighting signals W1 through Wm, respectively; an adder for addingoutput signals of the multipliers; and a control signal generatorcircuit for generating the control signals CTL1, CTL2, CTL3, and W1through Wm, wherein the switching unit is controlled in such a way that,when a signal stored in the line memory Mx (1≦×≦ n, where x is aninteger) which remains after the selection by the selector is the oldestsignal out of the line memories M1 through Mn, vertical transfer(scanning) of the solid state image pickup element is effected so that anew one-line sin SOnew is generated and then the signal SOnew is writteninto the line memory Mx.

In the image pickup apparatus with an interpolation function asdescribed above, there has been a problem that frequency responsecharacteristics may deteriorate in the interpolation process andtherefore affect the image quality.

DISCLOSURE OF THE INVENTION

The present invention is intended to improve the deterioration infrequency response characteristic when in the process of interpolationin the conventional image pickup apparatus having an interpolationfunction and thereby to obtain high-quality interpolation-processedimages, having a first objective to reduce the deterioration infrequency response characteristic in the process of interpolationhorizontal line formation found in the conventional electronic zoomfunction circuit.

A second objective is to generate frame still images and producehigh-quality still image outputs in an image pickup apparatus having astill image generation function. In order to achieve the aboveobjectives, the present invention comprises, in an image pickupapparatus, a plurality of solid state image pickup elements forobtaining different three chrominance signals; a vertical phase shiftsection for shifting a vertical phase of some of the three chrominancesignals obtained from the image pickup apparatus by a certain pitch; anda coefficient generating auxiliary circuit for changing an interpolationcoefficient for an interpolation process according to the shift of thephase,

wherein, with the present arrangement, the three chrominance signalsobtained from the plurality of solid state image pickup elements are outof phase in terms of a vertical phase by the vertical phase shiftsection, and therefore, when subjected to the interpolation process,deterioration in vertical frequency response characteristic due to theinterpolation process differs among the three chrominance signals,thereby reducing particularly deterioration in vertical frequencyresponse characteristic of the output line interpolated at around thecenter of two input lines, and thus, allowing to reduce thedeterioration in the vertical sharpness when in horizontal lineinterpolation.

Also, the vertical phase shift section may be replaced with at least oneimage pickup element drive circuit for driving the solid state imagepickup element and a drive control circuit for controlling the imagepickup element drive circuit, where also the same effects as above canbe obtained.

Further, by constituting so that the vertical phase shift section isreplaced with at least one image pickup element drive circuit fordriving the solid state image pickup element, a drive control circuitfor controlling the image pickup element drive circuit, and a verticalinterpolation selector circuit, wherein, by the vertical interpolationselector circuit, when in the vertical interpolation on, the drivecontrol circuit controls the image pickup element drive circuit so thatthe vertical phase of the three chrominance signals will be out ofphase, and when in the vertical interpolation off, the drive controlcircuit controls the image pickup element drive circuit so that thethree chrominance signals will be coincident in vertical phase, wherebywhen in the vertical interpolation on, deterioration in verticalsharpness during the horizontal line interpolation process can bereduced, and when the interpolation process is not effected, by keepingthe vertical interpolation off, three chrominance signals havingvertical phase coincident can be obtained, thus high resolution videoimages free from deterioration in vertical resolution can be obtained.

Still further, by adding a filter circuit for controlling frequencycharacteristics of signals, it becomes possible to remove false signalsfollowing to vertical phase shift.

Yet further, by adding a frame calculation circuit for obtaining apseudo frame signal from the three chrominance signals and aninterpolation circuit for obtaining an interpolation horizontal linesignal from an output signal of the frame calculation circuit, itbecomes possible to obtain an interpolation horizontal line by theinterpolation circuit from a pseudo frame signal generated from thethree chrominance signals shifted in vertical phase by the verticalphase shift section, thereby allowing to reduce deterioration invertical sharpness at the time of the horizontal line interpolation.

Yet further, by adding a frame still image circuit for obtaining a framestill image from an output signal of the frame calculation circuit, itbecomes possible to obtain a high-quality frame still image from apseudo frame signal generated from the three chrominance signals shiftedin vertical phase by the vertical phase shift section.

Yet further, by adding a vertical interpolation selector circuit to theabove arrangement, and by the vertical interpolation selector circuit,when in the vertical interpolation on, the drive control circuitcontrols the image pickup element drive circuit so that the threechrominance signals will be out of phase in terms of vertical phase, andwhen in the vertical interpolation off, the drive control circuitcontrols the image pickup element drive circuit so that the threechrominance signal will be coincident in vertical phase, it becomespossible to obtain three chrominance signals different in vertical phasewhen in the vertical interpolation on, allowing to obtain a high-qualityframe still image from a pseudo frame signal generated from the threechrominance signals different in vertical phase from one another. Also,when the still image generation process is not effected, by turning thevertical interpolation off, it is possible to obtain three chrominancesignals coincident in vertical phase and thereby to obtain ahigh-resolution image free from deterioration in vertical resolution.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing an arrangement of an image pickupapparatus with a horizontal line interpolation function according to afirst embodiment;

FIG. 2(a) is a schematic view showing an operation of phase shiftsections 4, 5 in the first embodiment;

FIG. 2(b) is an arrangement example of a color separation prism in thefirst embodiment;

FIG. 3 is a schematic view showing a concrete arrangement of aninterpolation circuit 14 in the first embodiment;

FIG. 4(a), FIG. 4(b), FIG. 4(c), and FIG. 4 (d) are views for explainingthe method of interpolation by the interpolation circuit 14 in the firstembodiment;

FIG. 5 is a schematic view showing an operation of the phase shiftsections 4, 5 when p1=p2=1/2 in the first embodiment;

FIG. 6(a), FIG. 6(b), FIG. 6(c), and FIG. 6 (d) are views for explainingthe method of interpolation by the interpolation circuit 14 whenp1=p2=1/2 in the first embodiment;

FIG. 7 is a block diagram showing the arrangement of an image pickupapparatus with a horizontal line interpolation function according to asecond embodiment;

FIG. 8 is a schematic view showing an operation of the phase shiftsection 4 in the second embodiment;

FIG. 9 is a view showing an arrangement of a light-receiving surface ofa solid state image pickup element 2 in the second embodiment;

FIG. 10 is an arrangement example of a two-color separation prism in thesecond embodiment;

FIG. 11(a), FIG. 11(b), FIG. 11(c), and FIG. 11(d) are views forexplaining the method of interpolation by the interpolation circuit 14in the second embodiment;

FIG. 12 is a block diagram of an image pickup apparatus with ahorizontal line interpolation function according to a third embodiment;

FIGS. 13(a) and 13(b) are explanatory views of a frame storage drivecontrol of an image pickup element of an image pickup apparatus with ahorizontal line interpolation function in the third, seventh, and eighthembodiments;

FIGS. 14(a) and 14(b) are an explanatory views of a field storage drivecontrol of an image pickup element of an image pickup apparatus with ahorizontal line interpolation function in the third, seventh, and eighthembodiments;

FIGS. 15(a) and 15(b) are explanatory views of a signal processing ofthe image pickup apparatus with a horizontal line interpolation functionin the third embodiments;

FIGS. 16(a) and 16(b) are constitutional diagrams of a digital signalprocessing circuit of the image pickup apparatus with a horizontal lineinterpolation function in the third and seventh embodiments;

FIGS. 17(a) 17(b) and 17(c) are explanatory views of a false chrominancesignal removal in the image pickup apparatus with a horizontal lineinterpolation function in the third embodiment;

FIG. 18 is a constitutional diagram of an interpolation function portionof the image pickup apparatus with a horizontal line interpolationfunction in the third embodiment;

FIG. 19 is a block diagram of an image pickup apparatus with ahorizontal line interpolation function in a fourth embodiment;

FIG. 20 is a block diagram of an image pickup apparatus with ahorizontal line interpolation function in a fifth embodiment;

FIG. 21 is a constitutional diagram of a vertical electronic zoomfunction of the image pickup apparatus with a horizontal lineinterpolation function in the fifth embodiment;

FIG. 22 is a constitutional diagram of vertical interpolation of theimage pickup apparatus with a horizontal line interpolation function inthe fifth embodiment;

FIG. 23 is a block diagram showing a digital signal processing sectionof an image pickup apparatus with a horizontal line interpolationfunction in a sixth embodiment;

FIG. 24 is a block diagram showing a digital signal processing circuitof the image pickup apparatus with a horizontal line interpolationfunction in the sixth embodiment;

FIG. 25 is a first explanatory view of horizontal line interpolation inthe image pickup apparatus with a horizontal line interpolation functionin the sixth and seventh embodiments;

FIG. 26 is a second explanatory view of horizontal line interpolation inthe image pickup apparatus with a horizontal line interpolation functionin the sixth embodiment;

FIG. 27 is a third explanatory view of horizontal line interpolation inthe image pickup apparatus with a horizontal line interpolation functionin the sixth embodiment;

FIG. 28 is an explanatory view of an output signal of the digital signalprocessing circuit in the image pickup apparatus with a horizontal lineinterpolation function in the sixth embodiment;

FIG. 29 is a constitutional diagram of an electronic zoom circuit in theimage pickup apparatus with a horizontal line interpolation function inthe sixth embodiment;

FIG. 30 is a block diagram of an image pickup apparatus with ahorizontal line interpolation function in the seventh embodiment;

FIGS. 31(a) and 31(b) are explanatory views of signal processing of theimage pickup apparatus with a horizontal line interpolation function inthe seventh embodiment;

FIG. 32 is a block diagram showing the digital signal processing circuitof the image pickup apparatus with a horizontal line interpolationfunction in the seventh embodiment;

FIG. 33 is a second explanatory view of a digital signal processingcircuit of the image pickup apparatus with a horizontal lineinterpolation function in the seventh embodiment;

FIG. 34 is an explanatory view of an output signal of the digital signalprocessing circuit of the image pickup apparatus with a horizontal lineinterpolation function in the seventh embodiment;

FIGS. 35(a), 35(b) and 35(c) are second explanatory views of an outputsignal of the digital signal processing circuit of the image pickupapparatus with a horizontal line interpolation function in the seventhembodiment; and

FIG. 36 is a block diagram of an image pickup apparatus with ahorizontal line interpolation function in the eighth embodiment.

BEST MODE FOR IMPLEMENTING THE INVENTION

[1] <First embodiment: Basic construction (A construction having threesolid state image pickup elements)>

FIG. 1 shows a block diagram of an image pickup apparatus with ahorizontal interpolation function in the first embodiment. In FIG. 1,solid state image pickup elements 1, 2, 3 are image pickup elements forobtaining G, R, B of the three chrominance signals, respectively, wherethe solid state image pickup element 1 obtains a G signal, the solidstate image pickup element 2 obtains an R signal, and the solid stateimage pickup element 3 obtains a B signal. A vertical phase shiftsection 4 is included in the figure for schematically showingimplementation of phase shift of a chrominance signal (where 2 and 4 arecombined to form an image pickup section 6), and a vertical phase shiftsection 5 is included for schematically showing implementation of phaseshift of a chrominance signal (where 3 and 5 are combined to form animage pickup section 7). It is assumed that, as for p1 and p2, aninterval between lines of a video signal to be interlace-scanned isstandardized to 1. Analog signal processing circuits 8, 9, 10 arecircuits for performing analog signal processing on the threechrominance signals obtained by the solid state image pickup elements 1,2, 3; analog-to-digital converter circuits 11, 12, 13 are circuits forconverting the three chrominance signals processed by the analog signalprocessing circuits 8, 9, 10, respectively, from analog to digital form;and an interpolation circuit 14 is a circuit for performinginterpolation on the three chrominance signals converted to digital formby the analog-to-digital converter circuits 11, 12, 13. A matrix circuit16 is a circuit for performing matrix operation on the three chrominancesignals interpolated by the interpolation circuit 14 to synthesize aluminance signal and two color-difference signals, and an encodercircuit 17 is a circuit for obtaining an NTSC signal from the luminanceand color-difference signals obtained by the matrix circuit 16. An imagepickup element drive circuit 18 is a circuit for driving the solid stateimage pickup element 1. A system control circuit 19 is a circuit forcomprehensively controlling the vertical phase shift sections 4, 5, andthe interpolation circuit 14, and the image pickup element drive circuit18.

FIG. 2(a) shows an operation of the vertical phase shift sections 4, 5,schematically representing that the solid state image pickup elements 2,3 are disposed so as to be shifted vertically with respect to the solidstate image pickup element 1 for performing the phase shift of thechrominance signals. As shown in FIG. 2(a), the solid state image pickupelements 2, 3 are located so as to be shifted with respect to the solidstate image pickup element 1 by p1, p2 (0≦p1<1, 0≦p2<1), respectively.

FIG. 2(b) is a constitutional diagram of a color separation prism forseparating G, R, and B signals, in the optical system of the imagepickup apparatus of the present embodiment. With the vertical length ofthe image pickup element set at 2L, the solid state image pickupelements 2, 3 are shifted in position vertically by p1 and p2,respectively.

FIG. 3 shows an example of a theoretical construction of theinterpolation circuit 14 of FIG. 1. One-line memories 20, 21, 22 arememories for storing 1-line portion of a video signal fed to theinterpolation circuit, where one of the three is used for writing, twoare used for reading, and the memories to be used for writing andreading are replaced with one another in turn by a switching unit 28 anda selector 29, so that real time processing can be implemented. Acoefficient generating auxiliary circuit 27 sends to a coefficientgenerator 26 information of vertical positional shift amount of thethree solid state image pickup elements (i.e. amount of vertical phaseshift of three chrominance signals). The coefficient generator circuit26 generates an interpolation coefficient to be used for interpolationcalculation based on the information from the coefficient generatingauxiliary circuit 27. Multipliers 23, 24 multiply signals derived fromthe 1-line memories with an interpolation coefficient obtained from thecoefficient generator circuit 26, and an adder 25 adds up the two linesignals multiplied with the interpolation coefficient. In addition, thewhole construction of the interpolation circuit 14 as shown in FIG. 3can be built up by providing the circuit construction shown in FIG. 3 toeach of the three chrominance signals (R, G, B).

The image pickup apparatus with a horizontal line interpolation functionof the present embodiment constructed as described above is nowexplained on its operation.

A G signal obtained by the solid state image pickup element 1, an Rsignal obtained by the image pickup section 6, and a B signal obtainedby the image pickup section 7 are subjected to analog signal processingby the analog signal processing circuits 8, 9, 10, respectively, thenconverted from analog to digital signals by the analog-to-digitalconverter circuits 11, 12, 13, and transmitted to the interpolationcircuit 14. The G, R, and B signals transmitted to the interpolationcircuit 14 are subjected to the respective interpolation processing, andthis interpolation processing is here explained with reference to FIG.4(a) through FIG. 4(d). First considered is the case where the solidstate image pickup elements 2, 3 are shifted with respect to the solidstate image pickup element 1 downward thereof (i.e. in the direction inwhich line number increases). When a line at distances of w from the kthline and 1-w from the (k+1)th line of a G signal is synthesized by theinterpolation (where it is assumed that line width is standardized to 1and that 0≦w<1), if 0≦w<p1, then the interpolation line to besynthesized from the R signal will be a line at distances of 1-p1+w fromthe (k-1)th line and p1-w from the kth line of the R signal, taking intoconsideration the phase shift with respect to the G signal. This is thesame also with the B signal, and if 0≦w<p2, then the interpolation lineto be synthesized from a B signal will be a line at distances of 1-p2+wfrom the (k-1)th line and p2-w from the kth line of the B signal (FIG.4(a)), taking into consideration the phase shift with respect to the Gsignal. Meanwhile, if p1≦w<1, then the interpolation line to besynthesized from an R signal will be a line at distances of w-p1 fromthe kth line and 1+p1-w from the (k+1)th line of the R signal, takinginto consideration the phase shift with respect to the G signal. This isthe same also with the B signal, if p2≦w<1, then the interpolation lineto be synthesized from a B signal will be a line at distances of w-p2from the kth line and 1+p2-w from the (k+1)th line of the B signal,taking into consideration the phase shift with respect to the G signal(FIG. 4(b)). Next considered is the case where the solid state imagepickup elements 2, 3 are shifted with respect to the solid state imagepickup element 1 upward (i.e. in the direction in which line numberdecreases). When a line at distances of w from the kth line and 1-w fromthe (k+1)th line of the G signal is synthesized by the interpolation, if0≦w<p1, then the interpolation line to be synthesized from an R signalwill be a line at distances of w+p1 from the kth line and 1-p1-w fromthe (k+1)th line of the R signal, taking into consideration the phaseshift with respect to the G signal. This is the same also with the Bsignal, if 0≦w<p2, then the interpolation line to be synthesized from aB signal will be a line at distances of w+p2 from the kth line and1-p2-w from the (k+1)th line of the B signal, taking into considerationthe phase shift with respect to the G signal (FIG. 4(c)). Meanwhile, ifp1≦w<1, then the interpolation line to be synthesized from an R signalwill be a line at distances of -1+p1+1 from the (k+1)th line and 2-p1-wfrom the (k+2)th line of the R signal, taking into consideration thephase shift with respect to the G signal. This is the same also with theB signal, and if p2≦w<1, then the interpolation line to be synthesizedfrom a B signal will be a line at distances of -1+p2+w from the (k+1)thline and 2-p2-w from the (k+2)th line of the B signal, taking intoconsideration the phase shift with respect to the G signal (FIG. 4(d)).

With an interpolation line synthesized in this way, an interpolationsignal in the same phase can be synthesized from three chrominancesignals shifted in phase from one another, and since deterioration invertical frequency response characteristic due to interpolation differsamong the three signals, in the case where the video signal is taken asa whole of the three chrominance signals (i.e. for example when aluminance signal synthesized from G, R, and B signals by matrixoperation is considered), deterioration in frequency response can bereduced, so that deterioration in vertical sharpness at the time ofhorizontal line interpolation can be reduced. For example, a case wherep1=p2=[1/2 line portion of a video signal to be interlace-scanned] isdescribed with reference to FIG. 5 and FIGS. 6(a) through (d). FIG. 5shows an operation of the vertical phase shift sections 4, 5,schematically representing that the solid state image pickup elements 2,3 are so arranged as to be shifted with respect to the solid state imagepickup element 1 vertically by a 1/2 line, in order to implement thephase shift of chrominance signals. First discussed is the case wherethe solid state image pickup elements 2, 3 are shifted with respect tothe solid state image pickup element 1 downward (i.e. in the directionin which line number increases). When a line at distances of w from thekth line, (1/2)+w from the (k+1)th line and 1-w from the kth line of theG signal is synthesized by the interpolation (where it is assumed thatline width is standardized to 1 and that 0≦w<1), if 0≦w<1, then theinterpolation line to be synthesized from an R signal will be a line atdistances of (1/2)+w from the (k-1)th line and (1/2)-w from the kth lineof the R signal, taking into consideration the phase shift with respectto the G signal. This is the same also with the B signal, and if 0<w<p2,then the interpolation line to be synthesized from a B signal will be aline at distances of (1/2)+w from the (k-1)th line and (1/2)-w from thekth line of the B signal, taking into consideration the phase shift withrespect to the G signal (FIG. 6(a)). Meanwhile, if p1≦w<1, then theinterpolation line to be synthesized from an R signal will be a line atdistances of (-1/2)+w from the kth line and (3/2)-w from the (k+1)thline of the R signal, taking into consideration the phase shift withrespect to the G signal. This is the same also with the B signal, ifp2≦w<1, then the interpolation line to be synthesized from a B signalwill be a line at distances of -1/2)+w from the kth line and (3/2)-wfrom the (k+1)th line of the B signal, taking into consideration thephase shift with respect to the G signal (FIG. 6(b)). Next discussed isthe case where the solid state image pickup elements 2, 3 are shiftedwith respect to the solid state image pickup element 1 upward (i.e. inthe direction in which line number decreases). When an interpolationline at distances of w from the kth line and 1-w from the (k+1)th lineof the G signal is synthesized by the interpolation, if 0≦w<p1, then theinterpolation line to be synthesized from an R signal will be a line atdistances of (1/2)+w from the kth line and (1/2)-w from the (k+1)th lineof the R signal, taking into consideration the phase shift with respectto the G signal. This is the same also with the B signal, and if 0≦w<p2,then the interpolation line to be constituted from a B signal will be aline at distances of (1/2)+w from the kth line and (1/2)-w from the(k+1)th line of the B signal, taking into consideration the phase shiftwith respect to the G signal (FIG. 6(c)). Meanwhile, if p1≦w<1, then theinterpolation line to be synthesized from an R signal will be a line atdistances of (-1/2)+w from the (k+ 1)th line and (3/2)-w from the(k+2)th line of the R signal, taking into consideration the phase shiftwith respect to the G signal. This is the same also with the B signal,and if p2 ≦w<1, then the interpolation line to be synthesized from a Bsignal will be a line at distances of (-1/2)+w from the (k+1)th line and(3/2)-w from the (k+2)th line of the B signal, taking into considerationthe phase shift with respect to the G signal (FIG. 6(d)). In the case asdescribed above, when such an interpolation process for w =0.5 with thelargest deterioration in vertical frequency response characteristic isdone on the G signal, neither the R nor the B signal is subjected to theinterpolation process and such a state can be maintained that thereoccurs no deterioration in vertical frequency response characteristic,and also, when an interpolation process for w=0 is done on the G signal,the R and B signals are interpolated with the largest deterioration invertical frequency response characteristic but the G signal is notinterpolated, so that such a state can be maintained that there occursno deterioration in vertical frequency response characteristic, andtherefore, in the case where the video signal is viewed as a whole ofthe three chrominance signals (i.e. for example when a luminance signalsynthesized from the G, R, and B signals by matrix operation isconsidered), deterioration in vertical sharpness at the time ofhorizontal line interpolation can be reduced.

In addition, for such cases where the solid state image pickup element 2is shifted upward with respect to the solid state image pickup element 1while the solid state image pickup element 3 is shifted downward, or thesolid state image pickup element 2 is shifted downward with respect tothe solid state image pickup element 1 while the solid state imagepickup element 3 is shifted upward, it is apparently also possible inthe present embodiment to implement interpolations by combining theinterpolations as described with reference to FIG. 4(a) through FIG.4(d).

Further, in the present embodiment, the solid state image pickup elementdrive circuits for the solid state image pickup elements 2, 3 areincluded in the vertical phase shift sections 4, 5 and undergo thecontrol of the system control circuit 19.

[2] <Second embodiment: Basic construction (A construction having twosolid state image pickup elements)>

Next, a second embodiment is described.

FIG. 7 shows a block diagram of an image pickup apparatus with ahorizontal interpolation function in the second embodiment.

In the figure, solid state image pickup elements 31 and 32, a verticalphase shift section 34, analog signal processing circuits 38, 39,analog-to-digital converter circuits 41, 42, 43, an interpolationcircuit 44, a matrix circuit 46, an encoder circuit 47, an image pickupelement drive circuit 48, and a system control circuit 49 arerespectively those which exhibit the same functions as the solid stateimage pickup elements 1, 2, the vertical phase shift section 4, theanalog signal processing circuits 8, 9, the analog-to-digital convertercircuits 11, 12, 13, the interpolation circuit 14, the matrix circuit16, the encoder circuit 17, the image pickup element drive circuit 18,and the system control circuit 19, respectively, of the first embodimentas shown in FIG. 1. Therefore, their detailed description is omitted.

In the same figure, the solid state image pickup elements 31, 32 areimage pickup elements for obtaining G, R and B of three chrominancesignals, respectively, where the solid state image pickup element 31obtains the G signal and the solid state image pickup element 32 obtainsR and B signals. The vertical phase shift section 34 is included in thefigure for schematically representing that phase shift of chrominancesignals is implemented (where 32 and 34 are combined to form an imagepickup section 36). In addition, it is assumed that line width of avideo signal to be interlace-scanned is standardized to 1. The G signalobtained by the solid state image pickup element 31 is subjected to thesame processing as in the first embodiment and then transmitted to theinterpolation circuit 44. The R and B signals obtained by the imagepickup section 36 are subjected to analog-to-digital signal processingat the analog signal processing circuit 39, thereafter separated into anR signal and a B signal by an R/B signal separation circuit 45, andsubjected to the same processing as in the first embodiment and thentransmitted to the interpolation circuit 44. It is noted that, since theinterpolation circuit 44 is the same as in the first embodiment (seeFIG. 3), the description is omitted.

FIG. 8 schematically represents that the solid state image pickupelement 32 is disposed so as to be shifted with respect to the solidstate image pickup element 31 in order to realize phase shift of achrominance signal. As shown in this FIG., the solid state image pickupelement 32 is disposed so as to be shifted with respect to the solidstate image pickup element 31 by p3 (0≦p3<1).

FIG. 9 is a view of a light-receiving surface of the solid state imagepickup element 32 and the light-receiving surface of the solid stateimage pickup element 32 is equipped with an R-B color stripe filter ofred (R) and blue (B) as shown in the figure, which arrangement allows Rand B signals to be obtained from the solid state image pickup element32. With the vertical length of the solid state image pickup element setto 2L, the solid state image pickup element 32 is shifted in verticalposition by p3.

FIG. 10 is a constitutional diagram of a two-color separation prism forseparating the G signal and the R and B signals, in the optical systemof the image pickup apparatus in the present embodiment. Among the inputlight, whereas light of R and B goes straight within the prism, light ofG is reflected by a dichroic layer as shown in the figure, so that G isseparated from R and B.

The image pickup apparatus with a horizontal line interpolation functionof the present embodiment constructed as described above is nowdescribed on its operation.

The G signal obtained by the solid state image pickup element 31 issubjected to analog signal processing by the analog signal processingcircuit 38, while the R and B signals obtained by the image pickupsection 36 are subjected to analog signal processing by the analogsignal processing circuit 39 and moreover separated into an R signal anda B signal by the R-B signal separation circuit, and thereafterconverted from analog to digital signals by the analog-to-digitalconverter circuits 41, 42, and 43, and then transmitted to theinterpolation circuit 44. The G, R, and B signals transmitted to theinterpolation circuit 44 are subjected to their correspondinginterpolation processing and this interpolation processing is describedwith reference to FIG. 11(a) through FIG. 11 (d). First discussed is acase where the solid state image pickup element 32 is shifted downwardwith respect to the solid state image pickup element 31 (i.e. in thedirection in which line number increases). When a line at distances of wfrom the kth line and 1-w from the (k+1)th line of the G signal (whereit is assumed that line width is standardized to 1 and that 0≦w<1), if0≦w<p3, then the interpolation line to be synthesized from an R signaland a B signal will be a line at distances of 1-p3+w from the (k-1)thline and p3-w from the kth line of the R signal and the B signal, takinginto consideration the phase shift with respect to the G signal (FIG.11(a)). Meanwhile, if p3≦w<1, then the interpolation line to besynthesized from an R signal and a B signal will be a line at distancesof w-p3 from the kth line and 1+p3-w from the (k+1)th line of the Rsignal and the B signal, taking into consideration the phase shift withrespect to the G signal (FIG. 11(b)). Next discussed is the case wherethe solid state image pickup element 32 is shifted upward with respectto the solid state image pickup element 31 (i.e. in the direction inwhich line number decreases). When a line at distances of w from the kthline and 1-w from the (k+1)th line of the G signal is synthesized by theinterpolation process, if 0≦w<p3, then the interpolation line to besynthesized from an R signal and a B signal will be a line at distancesof w+p3 from the kth line and 1-p3-w from the (k+1)th line of the Rsignal and the B signal, taking into consideration the phase shift withrespect to the G signal (FIG. 11(c)). Meanwhile, if p3≦w<1, then theinterpolation line to be synthesized from an R signal and a B signalwill be a line at distances of -1+p3+w from the (k+1)th line and 2-p3-wfrom the (k+2)th line of the R signal and the B signal, taking intoconsideration the phase shift with respect to the G signal (FIG. 11(d)).

With an interpolation line synthesized in this way, an interpolationsignal in the same phase can be synthesized from the G signal and the Rand B signals shifted in phase from one another, and since deteriorationin vertical frequency response characteristic due to interpolationdiffers among the G signal and the R and B signals, when the videosignal is viewed as a whole of the three chrominance signals (i.e. forexample when a luminance signal synthesized from G, R, and B signals bya matrix operation is considered), deterioration in vertical frequencyresponse characteristic can be reduced, so that deterioration invertical sharpness at the time of horizontal line interpolation can bereduced. For example, if p3=[1/2 line portion of a video signal to beinterlace-scanned], then when such an interpolation process for w=0.5with the largest deterioration in vertical frequency responsecharacteristic is done, neither the R nor the B signal is subjected tointerpolation process and such a state can be maintained that thereoccurs no deterioration in vertical frequency response characteristic,and also, when an interpolation process for w=0 is done on the G signal,the R and B signals are interpolated with the largest deterioration invertical frequency response characteristic while the G signal is notinterpolated, so that such a state can be maintained that there occursno deterioration in vertical frequency response characteristic, andtherefore, when the video signal is viewed as a whole of the threechrominance signals (i.e. for example when a luminance signalsynthesized from the G, R, and B signals by matrix operation isconsidered), deterioration in vertical sharpness at the time ofhorizontal line interpolation can be reduced.

Also, with the arrangement of the present embodiment, the vertical shiftsection for phase shift of chrominance signals can be made one innumber, so that the whole construction can be simplified compared to thefirst embodiment.

Further, in the present embodiment, the solid state image pickup elementdrive circuit for the solid state image pickup element 32 is included inthe vertical phase shift section 34, which undergoes the control of thesystem control circuit 49.

Further, the present embodiment has been arranged such that the R and Bsignals obtained by the image pickup section 36, after subjected to theanalog signal processing circuit 39, are separated into an R signal anda B signal by an R-B signal separation circuit, however, the presentinvention is not limited to this, but may be such that the analog signalprocessing circuit 39 and the R-B signal separation circuit are unitedinto one.

Further, the present embodiment has been arranged such that an R-B colorstripe filter is used for the solid state image pickup element 32 toobtain the R signal and the B signal, but the present invention is notlimited to this.

In addition, in the first embodiment, in order to shift the phase ofthree chrominance signals, R, G, and B, it is proper, for example, thatwhen the solid state image pickup elements, when the solid state imagepickup element is secured in contact to the three-color separation prismprovided for obtaining three chrominance signals, are positioned so asto be shifted vertically different from the conventional fashion,however, the present invention is not limited to this, and it may beconceived, for example, that the three-color separation prism isarranged to have such an inside refractive index that the optical pathof the light is curved, whereby phase of the chrominance signals isshifted. Also, in the second embodiment, in order to shift the phase ofG, and R and B of three chrominance signals respectively, it is proper,for example, that when the image pickup element is secured in contact tothe two-color separation prism provided for obtaining two chrominancesignals (a G signal and an R and B signal), the image pickup element ispositioned so as to be shifted vertically which is different from theconventional fashion, but the present invention is not limited to this,and it may be conceived, for example, that the two-color separationprism is arranged to have such an inside refractive index that theoptical path of input light is curved, whereby phase of the chrominancesignals is shifted.

Also, in the first and second embodiments, when the phases of the threechrominance signals are shifted, shown is an arrangement that the R andB signals are shifted with respect to the G signal, but which is notlimited to this, and it may be considered such that the G and B signalsare shifted with respect to the R signal or the G and R signals areshifted with respect to the B signal.

Also, in the first and second embodiments, the three chrominance signalsare R, G, and B, but the present invention is not limited to this, andfor example, three chrominance signals of yellow, cyan, and magenta maybe used.

Further, in the first and second embodiments, regarding theinterpolation circuit, only control of line memories is described,however, in addition to this, control for reading from the image pickupelements or field memories is also necessitated and such control can beimplemented according to shift amount of phase of the respective signalsin the G, R, and B signals, but the present invention is not limited tothis.

Further, in the first and second embodiments, the interpolation circuitis arranged to have three line memories, but is not limited to this, andmay be considered an arrangement having four or more line memories. Andthe interpolation processing made by linear interpolation in the abovetwo embodiments, but the present invention is not limited to this, andas is apparent, if it is arranged to have four or more line memories, itbecomes possible to implement high-order interpolation processing suchas quadratic interpolation.

Further, in the first and second embodiments, the range of p1, p2, andp3 is arranged to be equal to or more than 0 and less than 1, but thepresent invention is not limited to this, and for example, it isapparent that if it is set that p1=1.5, the same effect as in the caseof p1 =0.5 can be obtained (similarly as to p2 and p3).

Further, in the first and second embodiments, there have been used anarrangement having both a matrix circuit and an encoder circuit, but thepresent invention is not limited to this, and for example, it may bearranged that only either one of the matrix circuit and the encodercircuit is incorporated or that neither of them is incorporated.

Further, in the first and second embodiments, the output of the encodercircuit is an NTSC signal, but the present invention is not limited tothis.

[3] <Third embodiment: A construction having a drive control circuit anda frequency band limiting filter>

FIG. 12 shows a block diagram of an image pickup apparatus with ahorizontal line interpolation function in a third embodiment of thepresent invention. In FIG. 12, numeral 101 denotes an image pickupelement section having a photoelectric conversion function; 102 denotesan image pickup element drive circuit corresponding to the image pickupelement section 101; 103 denotes a drive control circuit for controllingthe image pickup element drive circuit 102; 104 denotes an analog signalprocessing circuit for performing such processing as sampling andamplification on the output signal of the image pickup element 101; 105denotes an analog-to-digital converter circuit (hereinafter, referred toas A/D converter) for an output signal of the analog signal processingcircuit 102; 106 denotes a digital signal processing circuit forperforming generation of luminance signals, chrominance signals orcolor-difference signals, and the like from an A/D converted digitalsignal or performing RGB signal processing; 107 denotes a field memorycircuit for storing output signals of the digital signal processingcircuit 106; 108 denotes a field memory control circuit for controllingthe field memory circuit 107; 109 denotes an electronic zoom circuit forperforming interpolation and enlargement processing with an outputsignal of the field memory circuit 107; 110 denotes an electronic zoomcontrol circuit for controlling the electronic zoom circuit 109; 111denotes a system control circuit for comprehensively controlling thedrive control circuit 103, the field memory control circuit 108, and theelectronic zoom control circuit 110; and 112 denotes an encoder circuitfor obtaining an NTSC signal from R, G, and B.

The image pickup apparatus with a horizontal line interpolation functionof the present embodiment constructed as described above is nowdescribed about its operation. A plurality of output signals of R, G,and B outputted from the image pickup element section 101 are subjectedto analog signal processing and A/D conversion, thus formed into digitalsignals. These digital signals are subjected to RGB signal processing atthe digital signal processing circuit 106, and fed to the field memorycircuit 107. The signals fed to the field memory circuit 107 arearithmetically processed for interpolation by the field memory controlcircuit 108, the electronic zoom circuit 109, and the electronic zoomcontrol circuit 110.

FIG. 13 shows a frame storage drive control of the image pickup elementby the drive control circuit 103. In FIG. 13(a), shown is an outline ofa common interlace-reading drive control of the image pickup element,while in FIG. 13(b) shown is an outline of a reading drive control of R,G and B image pickup elements for obtaining R, G and B signals,respectively, which is an arrangement example of the image pickupelement section 101 in the present embodiment. In the frame storage modeas shown in FIG. 13(a), in the odd field, signals of pixels of alternateodd-numbered lines are read vertically out of the pixels in thephotosensitive section during field shift periods, and then, in the evenfield, signals of pixels of even-numbered lines are read, and thus,interline transfer is accomplished. In the present embodiment, as shownin FIG. 13(b), in the odd field, the R and B image pickup elements amongthe R, G and B image pickup elements read signals of pixels ofodd-numbered lines vertically, while the G image pickup element readssignals of pixels of even-numbered lines, and next, in the even field,the R and B image pickup elements among the R, G and B image pickupelements read signals of pixels of even-numbered lines vertically, whilethe G image pickup element reads signals of pixels of odd-numberedlines. In this way, odd/even reading by the R, G and B image pickupdevices is inversed between the R and B image pickup elements and the Gimage pickup element by frame storage drive control.

Nextly, FIG. 14 shows a field storage drive control of the image pickupelements. FIG. 14(a) shows an outline of a common interlace-readingdrive control of the image pickup elements, while FIG. 14(b) shows anoutline of a reading drive control of the R, G and B image pickupelements for obtaining R, G and B signals, which is another arrangementexample of the image pickup element section 101 in the presentembodiment. In the field storage mode as shown in FIG. 14(a), in the oddfield, signals of odd-numbered lines and signals of their succeedingeven-numbered lines are simultaneously added up (PDmix) and read frompixels of lines close to a horizontal transfer CCD (not shown), andnext, in the even field, with combination for the addition changed,signals of even-numbered lines and signals of their succeedingodd-numbered lines from bottom are simultaneously added up and read, andthus, interline transfer is accomplished. In the present embodiment, asshown in FIG. 14(b), in the odd field, out of the R, G and B imagepickup elements, the R and B image pickup elements perform the readingof the odd field as shown in FIG. 14(a), while the G image pickupelement performs the reading of the even field, and next, in the evenfield, out of the R, G and B image pickup elements, the R and B imagepickup elements perform the reading of the even field, while the G imagepickup device performs the reading of the odd field. In this way,odd/even PDmix reading by the R, G and B image pickup elements isinversed between the R and B image pickup elements and the G imagepickup element by the field storage drive control.

As shown in FIG. 13 and FIG. 14 above, by inversing the odd/even readingbetween the R and B image pickup elements and the G image pickupelement, the resulting R and B signals and the G signal are shifted intheir spatial position (phase) by a 1/2 line (line width in one field).

As described above, according to the present embodiment, the provisionof the drive control circuit makes it possible to provide an imagepickup apparatus which does not necessitate an assembly process ofstrict precision that, in order to shift the phase of three signals, R,G, and B, the solid state image pickup elements are secured in contactto a three-color separation prism provided for obtaining threechrominance signals or a two-color separation prism so as to bevertically shifted in position, and which reduces the totalmanufacturing cost including manufacturing equipment and time, and whichalleviates deterioration in vertical frequency response characteristicdue to interpolation as well as reduces deterioration in verticalsharpness due to horizontal line interpolation.

Nextly, the method of signal processing for the resulting R, G, and Bchrominance signals is described below.

FIG. 15 shows an outline of the signal processing. FIG. 15(a) shows acase where the interpolation process is not performed, while FIG. 15(b)shows a case where the interpolation process is performed. Since the Rand B signal and the G signal are shifted in phase by a 1/2 line (linewidth in one field) as shown in FIG. 15(a), it is required to be madecoincident in vertical phase for signal processing. Therefore, two linessucceeding with respect to the G signal are subjected to averagingprocess (interpolation process with interpolation coefficients of 1/2and 1/2), whereby the phases of the R, G and B signals are madecoincident. This is shown in (Equation 1). Also, in the case of aninterpolation process as shown in FIG. 15(b), if the phase shift betweenthe R and B signal and the G signal is p (=1/2 line), then theinterpolation coefficient can be represented by a function of w and p,in which case the resulting interpolation signal is shown in (Equation2):

Equation 1:

    R.sub.m =R.sub.n, B.sub.m =B.sub.n, G.sub.m =(G.sub.-1 +G.sub.n)/2

Equation 2:

    R.sub.m =w×R.sub.n+2 +(1-w)×R.sub.n

    B.sub.m =w×B.sub.n+2 +(1-w)×B.sub.n

    G.sub.m =(w+p)×G.sub.n+1 +(1-w-p)×G.sub.n-1

With an interpolation line synthesized in this way, an interpolationsignal in the same phase can be synthesized from chrominance signals ofdifferent two types of phase, and moreover, since deterioration infrequency response characteristic due to interpolation differs dependingon their phase, when viewing the video signal as a whole of the threechrominance signals (i.e. for example when a luminance signalsynthesized from G, R and B signals by matrix operation is considered),deterioration in frequency response characteristic can be reduced, sothat deterioration in vertical sharpness of the horizontal lineinterpolation signal can be reduced. For example, if p=[1/2 line of avideo signal to be interlace-scanned], then even when such aninterpolation process for w=0.5 with the largest deterioration infrequency response characteristic is done, either of the G signal andthe R and B signal is not subjected to interpolation process and such astate can be maintained that there occurs no deterioration in frequencyresponse characteristic, and therefore, the horizontal lineinterpolation signal is reduced in deterioration in vertical sharpnessas the whole of the video signal.

Although a case of control (CTL1) where the G signal is spatiallyshifted in position has been described above in connection with FIGS.13, 14 and 15, yet another control (CTL1) in which the R and B signal isspatially shifted in position is also possible. Further, such controlthat position of the R signal only (CTL3) or the B signal only (CTL4) isspatially shifted is possible. This spatial position control isdescribed below. A luminance (Y) signal is generated by R, G and Bsignals, and is represented by (Equation 3):

Equation 3:

    Y=0.3R+0.59G+0.11B

where a spatially shifted signal (Ya) and a spatially unshifted signal(Yb) contained in the luminance signal are represented by the followingequations (Equation 4) for the above cases of CTL1-CTL4, respectively:##EQU1##

Color-difference signals (R-Y and B-Y) are generated from the R, G and Bsignals, and are represented by (Equation 5):

Equation 5:

    R-Y=0.7R-0.59G-0.11B

    B-Y=0.3R-0.59G-0.89B

where a spatially shifted signal (Ca) and a spatially unshifted signal(Cb) contained in the individual color-difference signals arerepresented by the following (Equation 6) for the above cases ofCTL1-CTL4, respectively: ##EQU2##

In order to reduce deterioration in vertical sharpness of a horizontalline interpolation signal in the whole video signal, it is necessarythat Ya and Yb are approximately equal to each other and Ca and Cb areapproximately equal to each other, in (Equation 4) and (Equation 6).This makes it understood that CTL1 and CTL2 are proper for the spatialposition control. Accordingly, the following explanation is made to onlythe cases where the spatial position control is CTL1 and CTL2.

Next described is the circuit construction of the digital signalprocessing circuit 106 with reference to FIG. 12. FIG. 16 shows anexample of the circuit construction. FIG. 16(a) takes a case where thespatial position control is CTL1, while FIG. 16(b) takes a case of CTL2.Those which exhibit the same effects as in FIGS. 16(a) and (b) aredesignated by like numbers, their detail being omitted. FIG. 16(a)comprises: a 1H memory 501 for delay of 1 horizontal line period of theG signal; an adder 502; a 1/2 amplifier 503 for gain control; aluminance signal matrix 504 for performing calculation as shown in(Equation 3); and a chrominance signal matrix 505 for performingcalculation as shown in (Equation 5). Similarly, FIG. 16(b) comprises: a1H memory 501 for delay of 1 horizontal line period of the R signal andthe B signal; an adder 502; a 1/2 amplifier 503 for gain control; aluminance signal matrix 504 for performing calculation as shown in(Equation 3); and a chrominance signal matrix 505 for performingcalculation as shown in (Equation 5). In the digital signal processingcircuit constructed as above, the provision of a vertical interpolationfunction having the 1H memory, i.e. a 2H line averaging circuit allowsthe chrominance signals to be made coincident in phase, in which stateluminance signal processing and chrominance signal processing aresubsequently performed. Further, from FIGS. 16(a) and (b), it can beunderstood that FIG. 16(a), i.e. the case where the spatial positioncontrol is CTL1 (G signal is shifted), is better suited for reduction incircuit scale.

On the other hand, in signal processing in the case where theinterpolation is not performed, phase adjustment with respect tochrominance signals which are shifted in spatial position (phase) asshown in FIG. 16 is done by LPF processing in the vertical direction,and therefore, high-band frequency characteristic would deteriorate.When the video signal is taken as a whole of the three chrominancesignals (i.e. for example when a luminance signal synthesized from G, Rand B signals by matrix operation is considered), the spatial positioncontrol CTL2 in which the R and B signal is shifted in phase as shown in(Equation 4) is superior in high-band frequency characteristic to CTL1in which the G signal is shifted, in the case of the luminance signalmatrix where the calculation as shown in (Equation 3) is performed. Alsoin the case of chrominance signal matrix where the calculation as shownin (Equation 5) is performed, the spatial position control CTL2 issuperior to CTL1 in high-band frequency characteristic as shown in(Equation 6). Thus, it can be understood that the spatial positioncontrol CTL2 is preferred for high-band frequency characteristic in thesignal processing in which interpolation processing is not performed.

Further, in the chrominance signal matrix in which color-differencesignals are generated, there would occur false signals in the highfrequency band due to different high-band frequencies of chrominancesignals. This point is described below. For example, when white color ispicked up, it is necessary for color-difference signals R-Y, B-Y to beat zero level. However, in the case of the spatial position controlCTL2, a G signal exists (G≠0) but an R/B signal does not exist (R=B=0)in the high frequency band, so that the color-difference signals R-Y,B-Y are other than at zero level as shown in (Equation 7) and thereforefalse chrominance signals will be generated. An arrangement example ofthe digital signal processing circuit 106 in FIG. 12 to remove thesefalse chrominance signals is shown in FIG. 17. In FIG. 17, numeral 601denotes a 1H memory for delay of 1 horizontal line period; 602 denotesan adder; 603 denotes a 1/2 amplifier for gain control; 604 denotes aluminance signal matrix for processing the calculation shown in(Equation 3); 605 denotes a chrominance signal matrix for processing thecalculation shown in (Equation 5); 606 denotes a phase adjuster circuitcomposed of the 601 to 603; 607 denotes a VLPF for attenuating high-bandfrequency components of the G signal; and 608 and 609 denote VLPFs forattenuating high-band frequency components of the color-differencesignals R-Y, B-Y.

Equation 7:

    R-Y|.sub.H =-0.59G

    B-Y|.sub.H =-0.59G

The signal processing circuit constructed as above is described below.Components 601 to 605 in FIG. 17 are the same as 501 to 505 in FIG. 16,and VLPFs 607, 608 and 609 only differ therefrom. Referring to FIG.17(a), the VLPF 607 attenuates the high-band frequency component of theG signal so that the component becomes equal to that of the R and Bsignal, thereby reducing false signals that would be generated in thehigh frequency band of color-difference signals. Similarly, referring toFIG. 17(b), the VLPFs 608, 609 attenuate high-band frequency componentsof color-difference signals, thereby reducing false signals that wouldbe generated in the high frequency band. In FIG. 17(c), where theeffects of FIG. 17(a) and (b) are added up, the high-band frequencycomponent of the G signal is attenuated and moreover high-band frequencycomponents of color-difference signals are attenuated, whereby falsesignals that would be generated in the high frequency band are reduced.In this way, the provision of the vertical LPFs allows false signalsgenerated in the high frequency band to be reduced, and thus, it becomespossible to provide an image pickup apparatus with a horizontal lineinterpolation function, free from false signals.

Nextly, an arrangement example of the interpolation function section asshown in FIG. 12, a block diagram of the image pickup apparatus with ahorizontal line interpolation function, is described in regard to onesignal with reference to FIG. 18. In the figure, numerals 701 to 704denote a vertical spatial position phase compensation section forperforming averaging process of 2 lines (interpolation processing withinterpolation coefficients of 1/2) for vertical phase adjustment asshown in FIG. 5, where 701 denotes a 1H memory; 702 denotes an adder;703 denotes a 1/2 amplifier; and 704 denotes a phase compensationcircuit composed of these components. Also, 705 denotes a field memory,and 706 denotes a switching unit for changing over write operations of1H line memories 707 to 709 shown in the next. Numeral 710 denotes aselector 710 for selecting two out of the line memories; 711 and 712denote multipliers; 713 denotes an adder; 714 denotes a verticalinterpolation circuit composed of the 710 to 713; 715 denotes a latchcircuit for performing horizontal delay; 716 and 717 denote multipliers;718 denotes an adder; 719 denotes a horizontal interpolation circuitcomposed of the 715 to 718; and 720 denotes an electronic zoom circuitcomposed of the vertical interpolation circuit 714 and the horizontalinterpolation circuit 719.

In the interpolation function portion constructed as above, a signalcompensated for vertical phase shift by the phase compensation circuit704 is stored in the field memory, then subjected to control by thefield memory 705 (not shown) and control of three line memories withinthe electronic zoom circuit 720, and processed for interpolationcalculation of multiplication data w1 and w2 within the verticalinterpolation circuit 714 (where when w2=1-wl, a primary interpolationis effected). Next, within the horizontal interpolation circuit 719,interpolation calculation of multiplication h1 and h2 is performed(where when h2=1-h1, a primary interpolation is effected). However, thephase compensation circuit 704 is necessitated for signals shifted inphase vertically, and not for signals which are not shifted in phasevertically.

As described above, the provision of a phase compensation circuit, avertical interpolation circuit, and a horizontal interpolation circuitmakes it possible to implement the vertical and horizontal interpolationfunction in an image pickup apparatus with a horizontal lineinterpolation function in which phase is shifted vertically.

[4] <Fourth embodiment: A construction having a vertical interpolationselector circuit added>

FIG. 19 is a block diagram of an image pickup apparatus with ahorizontal line interpolation function representing a fourth embodiment.In the figure, 801 to 810 are the same as 101 to 110 in FIG. 12, andonly a vertical interpolation SW circuit 711 and a system controlcircuit 812 for comprehensively controlling the aforementioned circuitdiffer therefrom. The image pickup apparatus with a horizontal lineinterpolation function constructed in this way is described below,focused on the different points.

In FIG. 19, when interpolation operation is not performed, the verticalinterpolation SW circuit 811 is kept in an off state, where the imagepickup element section 801 performs a normal drive operation in whichthe R, G and B signals are coincident in vertical phase with one anothervia the system control circuit 812, the drive control circuit 803, andthe image pickup element drive circuit 802. On the other hand, when ahorizontal line is generated by the interpolation function, the verticalinterpolation SW circuit 811 is thrown into an on-state, where the imagepickup element section 801 performs a drive operation in which the R andB signal and the G signal are different in vertical phase, by means asshown in FIG. 13 and FIG. 14 for the third embodiment, via the systemcontrol circuit 812, the drive control circuit 803, and the image pickupelement drive circuit 802.

In the image pickup apparatus with a horizontal line interpolationfunction of the present embodiment constructed as above, in the case ofsignal processing in which the interpolation process is not performed,phase adjustment for chrominance signals is not necessitated because ofcoincident spatial position (phase), neither is required to performvertical LPF processing, and therefore, high-band frequencycharacteristic will not deteriorate. Also, in the case of signalprocessing where the interpolation process is performed, deteriorationin frequency response characteristic due to the interpolation processdiffers depending on their phase, as in the third embodiment, andtherefore, when the video signal is taken as a whole of the threechrominance signals (i.e. for example when a luminance signalsynthesized from G, R and B signals by matrix operation is considered),deterioration in frequency response characteristic can be reduced anddeterioration in vertical sharpness of the horizontal line interpolationsignal can also be reduced.

As described above, according to the present embodiment, with theprovision of a drive control circuit and a vertical interpolation SWcircuit, a high-resolution image free from deterioration in verticalresolution can be obtained when the interpolation process is notperformed, and an image with less deterioration in image quality can beobtained as a result of reducing the deterioration in vertical sharpnessof a horizontal line interpolation signal when the interpolation processis performed.

[5] <Fifth embodiment: "Vertical interpolation circuit+horizontalinterpolation circuit" construction>

FIG. 20 is a block diagram of an image pickup apparatus with ahorizontal line interpolation function representing a fifth embodiment.In the figure, 901 to 905 are the same as 101 to 105 in FIG. 12, anddiffers from in: a field memory circuit 906 for storing an output signalof an A/D converter circuit 905; a field memory control circuit 907 forcontrolling the field memory circuit 906; a vertical interpolationcircuit 908 for performing a vertical interpolation process using anoutput signal of the field memory circuit 906; a vertical interpolationcontrol circuit 909 for controlling the vertical interpolation circuit908; a digital signal processing circuit 910 for generating luminancesignals, chrominance signals and color-difference signals, or the likefrom output signals of the vertical interpolation circuit 908 orperforming RGB signal processing; a horizontal interpolation circuit 911for performing a horizontal interpolation process using output signalsof the digital signal processing circuit 910; a horizontal interpolationcontrol circuit 912 for controlling the horizontal interpolation circuit911; and a system control circuit 913 for comprehensively controllingthe above circuits.

The image pickup apparatus with a horizontal line interpolation functionof the present embodiment constructed as above is described on itsoperation. The present embodiment differs from the third embodimentlargely in that the electronic zoom circuit is divided into the verticalinterpolation circuit 908 and the horizontal interpolation circuit 911.Now the electronic zoom function is explained with reference to FIG. 21and FIG. 22.

FIG. 21 shows an arrangement example of the vertical electronic zoomfunction. In the figure, reference numerals 1001, 1002, 1003 denotefield memory circuits for storing R, B, and G signals, respectively;1004 denotes a field memory control circuit for controlling the fieldmemory circuit 1003 (G signal); 1005 denotes a field memory controlcircuit for controlling the field memory circuits 1001 (R signal) and1002 (B signal); 1006, 1007, and 1008 denote vertical interpolationcircuits for performing vertical interpolation and enlargementprocessing on the R, B, and G signals derived from the field memorycircuits 1001, 1002, and 1003, respectively; 1009 denotes a verticalinterpolation control circuit for controlling the vertical interpolationcircuit 1008; 1010 denotes a vertical electronic zoom control circuitfor controlling the vertical interpolation circuits 1006 and 1007; and1011 denotes a vertical direction control circuit incorporated in thesystem control circuit 913 in FIG. 20, for controlling the above controlcircuits.

FIG. 22 shows the vertical interpolation circuit. As shown in thefigure, 1109 represents the whole vertical interpolation circuit, where1101 through 1108 are the same as 706 through 713 in FIG. 18 of thethird embodiment and their description is omitted. Also, among the R, G,and B signals, the circuit construction is of the same and thedifference is in the fact that the multiplication coefficient(interpolation coefficient) from the vertical interpolation controlcircuit to multipliers is v1 and v2 for the R and B signals, and v3 andv4 for the G signal.

The vertical electronic zoom function constructed as above is describedon the case where the R and B signals are shifted in vertical phase. TheR and B signals are controlled by the field memory circuits 1001 and1002 and the field memory control circuit 1005, and further by thevertical interpolation circuits 1006 and 1007 and the verticalinterpolation control circuit 1010, so that a horizontal line isinterpolated by calculation with vertical interpolation coefficients v1and v2 which are determined from a vertical phase shift amount p (1/2line) and the vertical interpolation coefficient w, as shown for the Gsignal in FIG. 15(b). Also, the G signal is controlled by the fieldmemory circuit 1003 and the field memory control circuit 1004, andfurther by the vertical interpolation circuit 1008 and the verticalinterpolation control circuit 1009, so that a horizontal line isinterpolated by calculation with vertical interpolation coefficients v3and v4 which are determined from the vertical interpolation coefficientw, as shown for the R and B signal in FIG. 15(b). Next, after verticalinterpolation processing, signal processing for R, G, B signals or Y(luminance)/C (color) signals is performed by the digital signalprocessing circuit 910, and horizontal interpolation processing isperformed by the horizontal interpolation circuit 911. This horizontalinterpolation circuit can be constructed by the same circuit as thehorizontal interpolation circuit 719 of FIG. 18, its description beingomitted.

With the provision of the vertical interpolation circuits 1006 and 1007as described above, it is possible to simultaneously perform the phasecompensation function for the vertical phase shift p and the verticalinterpolation function for the interpolation coefficient w, whereby thecircuit for vertical phase compensation can be reduced, and therefore animage pickup apparatus with a horizontal line interpolation function canbe provided in smaller circuit scale.

[6] <Sixth embodiment: Pseudo frame construction>

FIG. 23 is a block diagram showing the digital signal processing sectionof an image pickup apparatus with a horizontal line interpolationfunction which represents a sixth embodiment. In the figure, referencenumeral 1201 denotes a digital signal processing circuit for obtainingtwo types of luminance signals Y1, Y2 and two types of chrominancesignals CTL1, CTL2 from the R, G, and B signals; 1202 through 1205denote field memories for storing the various signals; 1206 denotes afield memory control circuit for controlling the field memoriesmentioned above; 1207 denotes an electronic zoom circuit for performinginterpolation and enlargement using Y1, Y2, C1, and C2; and 1208 denotesa system control circuit for comprehensively controlling the digitalsignal processing circuit, the field memory control circuit, and theelectronic zoom circuit. Also, FIG. 24 is a block diagram showing aconstruction of the digital signal processing circuit 1201 of FIG. 23.In the figure, reference numeral 1301 denotes a line memory for storingsignals of a 1H period; 1302 denotes an adder; 1303 denotes a 1/2amplifier for performing gain control; 1304 denotes a selector circuitfor selecting two signals R1 and R2 out of the three signals dependingon information from the system control circuit; 1305, 1306, and 1307denote signal selector circuits each composed of the above 1301 through1304; 1308 denotes a Y1 matrix circuit for generating a luminance signalY1; 1309 denotes a Y2 matrix circuit for generating a luminance signalY2; 1310 denotes a C1 matrix circuit for generating a chrominance signalC1; 1311 denotes a C2 matrix circuit for generating a chrominance signalC2; and 1312 denotes a matrix circuit composed of the above 1308 through1311.

The image pickup apparatus with a horizontal line interpolation functionconstructed as above is described below on its operation with referenceto FIGS. 25, 26, and 27. In FIG. 24, in the signal selector circuits1305 through 1307, firstly three signals are generated from signals ofsuccessive two lines. This is illustrated in FIG. 25. FIG. 25 shows asignal generated from R, G and B signals where the R and B signal isshifted in vertical phase with respect to the G signal by a 1/2 line.For example, in the R and B signal, an interpolation signal of the(m-1)th line is generated from signals of the (n-2)th line and the nthline while an interpolation signal of the (m+1)th line is generated fromsignals of the nth line and the (n+2)th line, and similarly, in the Gsignal, an interpolation signal of the mth line is generated fromsignals of the (n-1)th line and the (n+1)th line while an interpolationsignal of the (m+2)th line is generated from signals of the (n+ 1)thline and the (n+3)th line. In this way, signals of three lines includinginterpolation signals are generated from signals of successive twolines. Next, in the selector circuit 1304 in FIG. 24, two signals areselected out of the above three signals. This is illustrated in FIG. 26and FIG. 27. FIG. 26 shows an interpolation line h1 generated from theR, G and B signals in which the R and B signal is shifted in verticalphase with respect to the G signal by a 1/2 line as in the FIG. 25,while FIG. 27 shows an interpolation line h2 generated in the samemanner. As shown in FIG. 26, signals required to generate theinterpolation line h1 between the (n-1)th line and the nth line,R_(m-1), G_(m-1), B_(m-1) and R_(m), G_(m), B_(m), are shown by(Equation 8), while luminance signals Y_(m-1), Y_(m) andcolor-difference signals (R-Y)_(m-1), (R-Y)_(m) and (B-Y)_(m-1),(B-Y)_(m) to be generated are shown by (Equation 9): ##EQU3##

Also, as shown in FIG. 27, signals required to generate theinterpolation line h2 between the nth line and the (n+1)th line, R_(m),G_(m), B_(m) and R_(m+1), G_(m+1), B_(m+1), are shown by (Equation 10),while luminance signals Y_(m), Y_(m+1) and color-difference signals(R-Y)_(m), (R-Y)_(m+1) and (B-Y)_(m), (B-Y)_(m+1) to be generated areshown by (Equation 11): ##EQU4##

By selecting two signals required to generate interpolation lines out ofthree signals generated from signals of successive two lines, it becomespossible to generate an interpolation signal at any arbitrary position.The above-described selection of two signals is implemented by theselector circuit depending on the control from the system controlcircuit, and matrix operation of Y_(m-1) for generation of theinterpolation line h1 and Y_(m) for generation of the interpolation lineh2 is implemented by the Y1 matrix circuit 1308, and matrix operation ofY_(m) for generation of the interpolation line h1 and Y_(m+1) forgeneration of the interpolation line h2 is implemented by the Y2 matrixcircuit 130. Similarly, matrix operation of (R-Y)_(m-1), (B-Y)_(m-1) forgeneration of the interpolation line h1 and (R-Y)_(m), (B-Y)_(m) forgeneration of the interpolation line h2 is implemented by the C1 matrixcircuit 1310, while with matrix operation of (R-Y)_(m), (B-Y)_(m) forgeneration of the interpolation line h1 and (R-Y)_(m+1), (B-Y)_(m+1) forgeneration of the interpolation line h2 is implemented by the C2 matrixcircuit 1311. Further, in the C1 and C2 matrices, (R-Y) signals and(B-Y) signals are outputted in time division after decimated. Outputsignals of the digital signal processing circuit 1201 of FIG. 23, whichperforms the above-described operation, is shown in FIG. 23. In FIG. 28,for example, (Y_(m-1),1) represents the luminance signal of the 1stpixel of the (m-1)th line. FIG. 28(a) diagrams an output signal ofinterpolation as shown in FIG. 26, where chrominance signals aretime-sequenced with color-difference signals decimated every two pixels.Outputs in FIG. 28(a) are signals of the (m-1)th and mth lines, whileoutputs in FIG. 28(b) are signals of the mth and (m+1)th lines.

Next, output signals of the digital signal processing circuit 1201 inFIG. 23 are respectively stored in the field memories 1202-1205 bycontrol of the field memory control circuit 1206, and thereaftersubjected to interpolation calculation in the electronic zoom circuit1207 using the above signals from the field memories. The interpolationcalculation is performed with signals of two lines in the positionalrelation of a frame signal. For example, in the case of FIG. 26,interpolation calculation is implemented by using signals of the (m-1)thline and the mth line which are in the positional relation of a framesignal, while in the case of FIG. 27, the interpolation calculation issimilarly performed by using signals of the mth line and the (m+1)thline, which are in the positional relation of a frame signal. Anconstruction example of this electronic zoom circuit is illustrated inFIG. 29. In the figure, reference numeral 1801 denotes a switching unit;1802, 1803 denote line memories; 1804 denotes a line memory controlcircuit; 1805 denotes a selector circuit; 1806, 1807 denote verticalline memory control circuits; 1808, 1809 denote multipliers for verticalinterpolation; 1810 denotes an adder; 1811 denotes a latch circuit forgiving horizontal delay; 1812, 1813 denote multipliers for horizontalinterpolation; 1814 denotes an adder; and 1815 denotes a horizontalinterpolation circuit. Operation of the electronic zoom circuit isdescribed below, focused on its different points from the electroniczoom circuit 720 of FIG. 18. In FIG. 29, signals of two lines, Y1 and Y2or C1 and C2, which are to be calculated for interpolation, are fed toperform vertical interpolation calculation with the respective signalsand vertical interpolation data. Therefore, the vertical line memorycontrol circuits 1806 and 1807 control the two line memories so that oneof the two is used for Write and the other is for Read.

As described above, according to the present embodiment, with theprovision of a digital signal processing circuit and an electronic zoomcircuit for generating Y1 and Y2 signals and C1 and C2 signals, it ispossible to generate an interpolation signal from luminance signals andcolor-difference signals of two lines which are in the positionalrelation of a frame signal having less deterioration in image quality,and therefore, deterioration in vertical sharpness of a horizontal lineinterpolation signal can be reduced, so that an image with lessdeterioration in image quality can be obtained. Further, since signalsof two lines necessary for interpolation are fed as inputs to theelectronic zoom circuit, the number of lines memories within theelectronic zoom circuit is small and the circuit scale can be reduced.

In addition, in the third, fourth, fifth, and sixth embodiments, drivecontrol of the image pickup devices has been performed in order to shiftthe phase of three chrominance signals, R, G, and B, but, it may beconceived, for example, that the image pickup device, when secured incontact to the three-color separation prism for obtaining threechrominance signals, are shifted in position vertically unlike theconventional fashion, in addition to the drive control, or that thethree-color separation prism is arranged so as to have such an insiderefractive index that the optical path of light is curved, whereby thephase of chrominance signals is shifted. Also, in this case, the rangeof p1, p2, and p3 has been arranged to be equal to or more than 0 andless than 1, but the present invention is not limited to this, and forexample, it is apparently possible to arrange that p1=1.5, in which casethe same effect as in the case of p1=0.5 can be obtained (this isapplicable also to p2 and p3).

Further, the third, fourth, fifth, and sixth embodiments have beendescribed only on the fact that R, G, and B signals are generated asoutputs in connection to the image pickup section, however, there may beconsidered a construction, of a three-plate type image pickup apparatushaving three solid state image pickup elements for obtaining R, G, and Bsignals, respectively, or of a two-plate type image pickup apparatushaving two image pickup elements one of which is to obtain a G signalwhile the other is to obtain R and B signals.

Further, the third embodiment has been described taking the case whereR, G, and B signals from the electronic zoom circuit are converted intoNTSC signals by the encoder circuit, while the fourth, fifth, and sixthembodiments have been described up to a state where the R, G, and Bsignals or Y and C signals are outputted from the electronic zoomcircuit, however, the signals may also be converted into NTSC signals orother signals.

Further, in the third, fourth, fifth, and sixth embodiments, the threesignals have been assumed to be R, G, and B, but the present inventionis not limited to this, and for example, three chrominance signals ofyellow, cyan, and magenta may be used.

Further, in the third, fourth, fifth, and sixth embodiments, theinterpolation circuit has been described with regard to only control ofline memories, but, in addition to this, control for reading from theimage pickup elements or field memories is also necessitated, but, suchcontrol can be implemented depending on the R, G, and B signals or Y andC signals, but the present invention is not limited to this.

Further, in the third, fourth, fifth, and sixth embodiments, theinterpolation process has been described in the case of linearinterpolation, but the present invention is not limited to this, and itis apparently possible to perform high-order interpolation processingsuch as quadratic interpolation by using three or more line memoryoutput signals.

Further, it is possible to use the contents of the third, fourth, fifth,and sixth embodiments described for the individual embodiments, incombination with one another. For example, although removal of falsesignals has been described in connection to only the third embodiment,yet this may be used in combination with the fourth, fifth, and sixthembodiments.

[7] <Seventh embodiment: A construction for forming a frame still image>

FIG. 30 is a block diagram of an image pickup apparatus with ahorizontal line interpolation function in a seventh embodiment.Referring to FIG. 30, reference numeral 2101 denotes an image pickupelement section having a photoelectric conversion function; 2102 denotesan image pickup element drive circuit for the image pickup elementsection 2101; 2103 denotes a drive control circuit for controlling theimage pickup element drive circuit 2102; 2104 denotes an analog signalprocessing circuit for performing processing such as sampling andamplification on output signals of the image pickup element 2101; 2105denotes an analog-to-digital converter circuit (hereinafter, referred toas A/D converter) for output signals of the analog signal processingcircuit 2102; 2106 denotes a digital signal processing circuit forperforming generation of luminance signals, chrominance signals andcolor-difference signals, or the like from A/D converted digital signalsor performing RGB signal processing; 2107 denotes a field memory circuitfor storing output signals of the digital signal processing circuit2106; 2108 denotes a field memory control circuit for controlling thefield memory circuit 2107; 2109 denotes a system control circuit forcomprehensively controlling the drive control circuit 2103 and the fieldmemory control circuit 2108; 2110 denotes an encoder circuit forobtaining television signals such as NTSC signals from output signals ofthe digital signal processing circuit 2106; and 2111 denotes an encodercircuit for obtaining television signals such as NTSC signals fromoutput signals of the field memory circuit 2107.

The image pickup apparatus with a horizontal line interpolation functionof the present embodiment constructed as above is described below on itsoperation. A plurality of output signals of R, G, and B fed from theimage pickup element section 2101 are subjected to analog signalprocessing and A/D conversion and thus formed into digital signals.These digital signals are processed for luminanoe signals (Y1, Y2) andchrominance signals (C1, C2) by the digital signal processing circuit2106, and fed to the field memory circuit 2107. The signals fed to thefield memory circuit 2107 are yielded as a frame still image output bythe field memory control circuit 2108.

FIG. 13 shows frame storage drive control of the image pickup elementsby the drive control circuit 2103. FIG. 13(a) shows an outline of thecommon interlace-reading drive control of the image pickup elements,while FIG. 13(b) shows an outline of the reading drive control of R, G,and B image pickup elements for obtaining each R, G, and B signals,which is an construction example of the image pickup element section2101 in the present embodiment. In the frame storage mode as shown inFIG. 13(a), in the odd field, signals of pixels of alternateodd-numbered lines are read vertically out of the pixels in thephotosensitive section during field shift periods, and next, in the evenfield, signals of pixels of even-numbered lines are read, and thus,interline transfer is accomplished. In the present embodiment, as shownin FIG. 13(b), in the odd field, the R and B image pickup elements outof the R, G and B image pickup elements read signals of pixels ofodd-numbered lines vertically, while the G image pickup element readssignals of pixels of even-numbered lines, and next, in the even field,the R and B image pickup elements out of the R, G and B image pickupelements read signals of pixels of even-numbered lines vertically, whilethe G image pickup element reads signals of pixels of odd-numberedlines. In this way, odd/even reading by the R, G and B image pickupelements is inversed between the R and B image pickup elements and the Gimage pickup element by the frame storage drive control.

Nextly, FIG. 14 shows field storage drive control of the image pickupelements. FIG. 14(a) outlines common interlace-reading drive control ofthe image pickup elements, while FIG. 14(b) outlines reading drivecontrol of the R, G and B image pickup elements for obtaining R, G and Bsignals, which is another arrangement example of the image pickupelement section 2101 in the present embodiment. In the field storagemode as shown in FIG. 14(a), in the odd field, signals of odd-numberedlines and signals of their succeeding even-numbered lines aresimultaneously added up (PDmix) and read from pixels of lines close to ahorizontal transfer CCD (not shown), and then in the even field, withcombination for the addition changed, signals of even-numbered lines andsignals of their succeeding odd-numbered lines as counted from bottomare simultaneously added up and read, so that the interline transfer isaccomplished. In the present embodiment, as shown in FIG. 14(b), in theodd field, out of the R, G and B image pickup elements, the R and Bimage pickup elements perform the reading of the odd field as shown inFIG. 14(a ), while the G image pickup element performs the reading ofthe even field, and then in the even field, out of the R, G and B imagepickup elements, the R and B image pickup elements perform the readingof the even field, while the G image pickup element performs the readingof the odd field. In this way, odd/even PDmix reading by the R, G and Bimage pickup elements is inversed between the R and B image pickupelements and the G image pickup element by the field storage drivecontrol.

As shown in FIG. 13 and FIG. 14 above, by inversing the odd/even readingbetween the R and B image pickup elements and the G image pickupelement, the resulting R and B signals and the G signal are shifted intheir spatial position (phase) by a 1/2 line (line width in one field).

As described above, according to the present embodiment, there isexemplified a case where a drive control circuit is provided to shiftthe phase of the three chrominance signals, R, G, and B. Other thanthis, the three chrominance signals R, G, and B can be shifted in phasealso by securing the solid state image pickup elements in contact to athree-color separation prism provided for obtaining three chrominancesignals or a two-color separation prism in such a way that the imagepickup elements are shifted in position vertically.

Nextly, the method of signal processing for the resulting R, G, and Bchrominance signals is described below.

FIG. 31 outlines the signal processing. FIG. 31(a) outlines processingfor obtaining a G signal coincident in phase with the R signal and the Bsignal, while FIG. 31(b) outlines processing for obtaining an R signaland a B signal coincident in phase with the G signal. Since the R and Bsignal and the G signal are shifted in phase by a 1/2 line (line widthin one field) as shown in FIG. 31, the signals need to be madecoincident in vertical phase for signal processing. Thus, in the case ofFIG. 31(a), two lines succeeding with respect to the G signal aresubjected to averaging process (interpolation process with interpolationcoefficients of 1/2), so that the phase of the R, G and B signals aremade coincident in phase with each other, while in the case of FIG.31(b), averaging process is effected on two lines succeeding withrespect to the R and B signal (interpolation process with interpolationcoefficients of 1/2), whereby the R, G, and B signals can be madecoincident in phase. This is shown in (Equation 12): ##EQU5##

Although a case of control (CTL1) where the G signal is spatiallyshifted in position has been described above in connection to FIGS. 13,14 and 31(a), yet another control (CTL2) in which the R and B signal isspatially shifted in position is also possible in FIG. 31(b). Further,such control that position of the R signal only (CTL3) or the B signalonly (CTL4) is spatially shifted is possible. This spatial positioncontrol is described below. A luminance (Y) signal is generated by R, Gand B signals, and is represented by Equation 3:

    Y=0.3R+0.59G+0.11B

where a spatially shifted signal (Ya) and a spatially unshifted signal(Yb) both contained in the luminance signal are represented by thefollowing equations (Equation 4) for the above cases of CTL1-CTL4,respectively: ##EQU6##

Color-difference signals (R-Y and B-Y) are generated from the R, G and Bsignals, and are represented by (Equation 5):

Equation 5:

    R-Y=0.7R-0.59G-0.11B

    B-Y=-0.3R-0.59G+0.89B

where a spatially shifted signal (Ca) and a spatially unshifted signal(Cb) both contained in the individual color-difference signals arerepresented by the following (Equation 6) for the above cases ofCTL1-CTL4, respectively: ##EQU7##

In order to obtain a pseudo frame video signal as a whole video signal,it is necessary for Ya and Yb to be approximately equal and for Ca andCb to be approximately equal, by (Equation 4) and (Equation 5). Thisfact makes it understood that CTL1 or CTL2 is proper for the spatialposition control. Accordingly, only cases of CTL1 and CTL2 spatialposition control are described hereinbelow.

Nextly, an example of circuit construction for the signal processing asshown in FIG. 31 is outlined in FIG. 16. FIG. 16(a) is a case where thespatial position control is CTL1, while FIG. 16(b) is a case where thespatial position control is CTL2. Those which exhibit the same effectsbetween FIGS. 16(a) and (b) are designated by like numerals and omittedin their description. FIG. 16(a) comprises: a 1H memory 501 for delay of1 horizontal line period of the G signal; an adder 502; a 1/2 amplifier503 for gain control; a luminance signal matrix 504 for performingcalculation as shown in (Equation 3); and a chrominance signal matrix505 for performing calculation as shown in (Equation 5). Similarly, FIG.16(b) comprises: a 1H memory 501 for delay of 1 horizontal line periodof the R an B signal; an adder 502; a 1/2 amplifier 503 for gaincontrol; a luminance signal matrix 504 for performing calculation asshown in (Equation 3); and a chrominance signal matrix 505 forperforming calculation as shown in (Equation 5). In the digital signalprocessing circuit constructed as above, the provision of a verticalinterpolation function having the 1H memory, i.e. a 2H line averagingcircuit allows the chrominance signals to be made coincident in phase,in which state luminance signal processing and chrominance signalprocessing are subsequently performed. Further, from FIGS. 16(a) and(b), it can be understood that FIG. 16(a), i.e. the case where thespatial position control is CTL1 (G signal is shifted), is better suitedfor reduction in circuit scale.

On the other hand, phase adjustment with respect to chrominance signalswhich are shifted in spatial position (phase) as shown in FIG. 16 isdone by LPF processing in the vertical direction, and therefore, in thiscase, high-band frequency characteristic would deteriorate. When thevideo signal is viewed as a whole of the three chrominance signals (i.e.for example when a luminance signal synthesized from G, R and B signalsby matrix operation is considered), the spatial position control CTL2 inwhich the R and B signal is shifted in phase as shown in (Equation 4) issuperior in high-band frequency characteristic to CTL1 in which the Gsignal is shifted, in the case of the luminance signal matrix where thecalculation as shown in (Equation 3) is performed. Also in the case ofchrominance signal matrix where the calculation as shown in (Equation 5)is performed, the spatial position control CTL2 is superior to CTL1 inhigh-band frequency characteristic as shown in (Equation 6). Thus, itcan be understood that the spatial position control CTL2 is preferredfor high-band frequency characteristic in the signal processing in whichinterpolation processing is not performed.

Further, in the chrominance signal matrix in which color-differencesignals are generated, there would occur false signals in the highfrequency band due to different high-band frequencies of chrominancesignals. For example when white color is picked up, it is necessary forcolor-difference signals R-Y, B-Y to be at zero level. However, in thecase of the spatial position control CTL2, a G signal exists (G≠0) butan R signal or a B signal does not exist (R=B=0) in the high frequencyband, so that the color-difference signals R-Y, B-Y are other than atzero level as shown in (Equation 7) and therefore false chrominancesignals will be generated. This point has been described in detail inthe third embodiment, its description being omitted.

Next described is the circuit construction of the digital signalprocessing circuit 2106 in FIG. 30, with reference to FIG. 32. In thefigure, reference numeral 2601 denotes a line memory for storing signalsof a 1H period; 2602 denotes an adder; 2603 denotes a 1/2 amplifier forperforming gain control; 2604 denotes a signal interpolation circuitcomposed of the 2601 to 2603; 2605 denotes a Y1 matrix circuit forgenerating a luminance signal Y1; 2606 denotes a Y2 matrix circuit forgenerating a luminance signal Y2; 2607 denotes a C1 matrix circuit forgenerating a chrominance signal C1; 2608 denotes a C2 matrix circuit forgenerating a chrominance signal C2; 2609 denotes a matrix circuitcomposed of the 2605 to 2608; 2610 denotes a luminance signal processingcircuit for performing processing such as aperture or coring on theluminance signals Y1 and Y2; and 2611 denotes a chrominance signalprocessing circuit for performing processing such as white balance andcolor regeneration on the chrominance signals C1 and C2.

The image pickup apparatus with a horizontal line interpolation functionconstructed as above is described below on its operation with referenceto FIGS. 25, 33, and 34. Referring to FIG. 32, in the signalinterpolation circuit 2604, an interpolation signal is generated fromsignals of successive two lines. This is illustrated in FIG. 25. FIG. 25shows a signal generated from R, G and B signals where the R and Bsignal is shifted in vertical phase with respect to the G signal by a1/2 line. For example, in the R and B signal, an interpolation signal ofthe (m+1)th line is generated from signals of the (n-2)th line and thenth line while an interpolation signal of the (m+1)th line is generatedfrom signals of the nth line and the (n+2)th line, and similarly, in theG signal, an interpolation signal of the mth line is generated fromsignals of the (n-1)th line and the (n+1)th line while an interpolationsignal of the (m+2)th line is generated from signals of the (n+1 )thline and the (n+3)th line. In this way, interpolation signals aregenerated from signals of successive two lines. Next, the matrix circuit2609 in FIG. 32 performs matrix signal processing with signalinterpolation circuit output signals of R, G, and B. FIG. 33 showsluminance signals Y_(m-1), Y_(m) . . . and color-difference signals(R-Y)_(m-1), (R-Y)_(m) . . . and (B-Y)_(m-1), (B-Y)_(m) . . . to begenerated from R, G, and B signals in which the R and B signals areshifted in vertical phase with respect to the G signal by a 1/2 line, asin FIG. 25. As shown in FIG. 33, chrominance signals R_(m-1), G_(m-1),B_(m-1) and R_(m), G_(m), B_(m) are shown by (Equation 8), whileluminance signals Y_(m-1), Y_(m) and color-difference signals(R-Y)_(m-1), (R-Y)_(m) and (B-Y)_(m-1), (B-Y)_(m) to be generated areshown by (Equation 9): ##EQU8##

The above matrix operation for Y_(m) -1 is performed by the Y1 matrixcircuit 2605, and the matrix operation for Y_(m) is performed by the Y2matrix circuit 2606. Similarly, the matrix operation for (R-Y)_(m-1),(B-Y)_(m) -1 is performed by the C1 matrix circuit 2607, and the matrixoperation for (R-Y)_(m), (B-Y)_(m) is performed by the C2 matrix circuit2608. Further, in the C1 and C2 matrices, (R-Y) signals and (B-Y)signals are outputted in time division after decimated. Output signalsof the digital signal processing circuit 2106 of FIG. 30 are shown inFIG. 34. In FIG. 34, for example, (Y_(m-1),1) represents the luminancesignal of the 1st pixel of the (m-1)th line. FIG. 34 shows a case wherechrominance signals are time-sequenced with color-difference signalsdecimated every two pixels. Outputs in FIG. 34 are a (m-1)th line signalas the Y1 signal and a mth line signal as the Y2 signal.

Next, output signals of the digital signal processing circuit 2106 inFIG. 30 are converted to a television signal (TV1) by the encodercircuit 2110 and outputted as a dynamic image. Also, output signals ofthe digital signal processing circuit 2106 are controlled by the fieldmemory control circuit 2108 so that images at any arbitrary moments arestored in the field memory circuit 2107, and outputted as a first fieldsignal composed of Y1 and C1 signals, and as a second field signalcomposed of Y2 and C2 signals and as a luminance signal Y3 and acolor-difference signal C3 in time sequence, and further, the Y3 signaland the C3 signal are converted to a television signal (TV2) by theencoder circuit 2111 and outputted as a still image.

This is explained with reference to FIG. 35. In FIG. 35, (a) showsspatial positions represented by the Y1 and C1 signals, (b) showsspatial positions represented by Y2 and C2 signals, and (c) showsspatial positions represented by Y1, Y2 and CTL1, CTL2 signals. The Y1and C1 signals in FIG. 35(a) and the Y2 and C2 signals in (b) areoutputted simultaneously every field from the digital signal processingcircuit 2106 as shown in FIG. 34. Thus, the above television signal(TV1) is composed of the Y1 signal and the C1 signal and outputted as adynamic image, and the Y3 and C3 signals form an output of Y1 and C1signals at positions of (a) as a first field signal, an output of Y2 andC2 signals at positions of (b) as a second field signal, where the firstand second field signals form a high-quality frame still image. Also,the television signal (TV2) combines the Y3 signal and the C3 signalinto a composite signal, where the first and second filed signals form ahigh-quality frame still image.

As described above, according to the present embodiment, with theprovision of a digital signal processing circuit for generating Y1 andY2 signals and C1 and C2 signals, a field memory circuit, and a fieldmemory control circuit, it is possible to generate luminance signals andcolor-difference signals in the positional relation of a frame signaland to obtain two field signals forming a frame signal of the same timepoint.

[8] <Eighth embodiment: a construction for forming a frame stillimage+vertical interpolation selector circuit added>

FIG. 36 is a block diagram of an image pickup apparatus with ahorizontal line interpolation function which shows an eighth embodiment.In the figure, 3101 through 3108 and 3110, 3111 are the same as 2101through 2108 and 2110, 2111 in FIG. 30, and a vertical interpolation SWcircuit 3112 and a system control circuit 3109 for comprehensivelycontrolling the circuits differ therefrom. The image pickup apparatuswith a horizontal line interpolation function constructed as above isdescribed below, focused on the different points.

Referring to FIG. 36, when still image generation process is notperformed, the vertical interpolation SW circuit 3112 is kept in an offstate, where the image pickup element section 3101 performs a normaldrive operation in which the R, G and B signals are coincident invertical phase with one another via the system control circuit 3109, thedrive control circuit 3103, and the image pickup element drive circuit3102. On the other hand, when still image processing is performed by thestill image generation function, the vertical interpolation SW circuit3112 is turned to an on-state, where the image pickup element section3101 performs a drive operation in which the R and B signal and the Gsignal are different in vertical phase, by means as shown in FIG. 13 andFIG. 14 for the seventh embodiment, via the system control circuit 3109,the drive control circuit 3103, and the image pickup element drivecircuit 3102.

In the image pickup apparatus with a horizontal line interpolationfunction of the present embodiment constructed as above, in the case ofsignal processing in which the still image generation process is notperformed, phase adjustment for chrominance signals is not necessitatedbecause of coincident spatial position (phase), and neither is requiredto perform vertical LPF processing, and therefore, high-band frequencycharacteristic will not deteriorate. Also, in the case of signalprocessing where the frame still image generation process is performed,the digital signal processing circuit can generate Y1, Y2 and C1, C2signals, while the field memory circuit and the field memory controlcircuit can generate frame still images, as in the seventh embodiment.

As described above, according to the present embodiment, with theprovision of a drive control circuit and a vertical interpolation SWcircuit, a high-resolution image free from deterioration in verticalresolution can be obtained when the still image process is notperformed, and two field signals forming a frame signal of the same timepoint can be obtained when the still image process is performed.

In addition, in the seventh and eighth embodiments, drive control of theimage pickup elements has been performed in order to shift the phase ofthree chrominance signals, R, G, and B, but it may be conceived, forexample, that the solid state image pickup elements, when secured incontact to the three-color separation prism for obtaining threechrominance signals, are shifted in position vertically unlike theconventional fashion, in addition to the drive control, or that thethree-color separation prism is arranged so as to have such an insiderefractive index that the optical path of light is curved, whereby thephase of chrominance signals is shifted. Also, in this case, the rangeof p1, p2, and p3 has been arranged to be equal to or more than 0 andless than 1, but the present invention is not limited to this, and forexample, it is apparently possible to arrange that p1=1.5, in which casethe same effect as in the case of p1=0.5 can be obtained (this isapplicable also to p2 and p3).

Further, the seventh and eighth embodiments have been described taking acase where the three signals are outputs of the solid state image pickupelements, but, the case is the same with signals stored in filedmemories or frame memories or the like, where means for shifting thevertical phase can perform memory read control.

Further, in the seventh and eighth embodiments, the image pickup elementsection has been described only in terms of its output of R, G, and Bsignals, but, as the construction therefor, there may be considered athree-plate type image pickup apparatus having three solid state imagepickup elements for obtaining R, G, and B signals respectively, and atwo-plate type image pickup apparatus having two image pickup elementsof which one element is for obtaining the G signal and the other elementfor obtaining the R signal and B signal.

Further, in the seventh and eighth embodiments, the three signals havebeen assumed to be R, G, and B, but the present invention is not limitedto this, and for example, three chrominance signals of yellow, cyan, andmagenta may be used.

Further, in the seventh and eighth embodiments, the field memory circuithas been described only on the function of recording images at anyarbitrary moments as well as producing an output as a still image bycontrol of the field memory control circuit, and otherwise, for examplethe recordable number of images can be determined depending on thememory capacity, and the function of instructing arbitrary moments forrecording or the way of updating images and the like can be implementedby the provision of switching function via the system control circuitand the field memory control circuit, and therefore, its description isomitted.

Further, the seventh and eighth embodiments have been described taking acase matching the current television signals of the interlacing systemas the form of output, however, it is also possible to output framedynamic signals (Y1, Y2 and C1, C2 signals) as a dynamic output matchingtelevisions of the non-interlacing system, or to output 1-frame image incompliance with the printer input system as an output for still images.

Yet further, the seventh and eighth embodiments have been describedtaking a case where Y1, Y2 signals and C1, C2 signals are outputted aspseudo frame signals (frame positional signals generated byinterpolation), however, the present invention, without being limited tothis, may be such that pseudo frame signals of R1, R2 and G1, G2 and B1,B2 are outputted by each of R, G, and B signals, or that Y signals areoutputted as pseudo frame signals and C signals are outputted as normalfield signals, taking into consideration sensitivity with respect toresolution of the human eye.

What is claimed is:
 1. An image pickup apparatus with a horizontalinterpolation function comprising:a plurality of solid state imagepickup elements for obtaining three chrominance signals C1, C2, and C3which are different from each other; a first vertical phase shiftsection for shifting a vertical phase of the chrominance signal C2 withrespect to the chrominance signal C1 by a specified pitch p1; a secondvertical phase shift section for shifting a vertical phase of thechrominance signal C3 by a specified pitch p2; coefficient generatormeans for generating:(1) a first interpolation coefficient w forinterpolating the chrominance signal C1 where 0≦w <1; (2) a secondcoefficient for interpolating the chrominance signal C2 from the firstinterpolation coefficient w and the specified pitch p1; (3) a thirdcoefficient for interpolating the chrominance signal C3 from theinterpolation coefficient w and the specified pitch p2; an interpolationcircuit for producing an interpolation horizontal line signal byinterpolating the chrominance signal C1 using the first interpolationcoefficient w, the chrominance signal C2 using the second coefficientand, the chrominance signal C3 using the third coefficient.
 2. The imagepickup apparatus with a horizontal line interpolation function asclaimed in claim 1, wherein said plurality of solid state image pickupelements for obtaining the different three chrominance signals C1, C2,and C3 comprises a first solid state image pickup element for obtainingthe chrominance signal C1, a second solid state image pickup element forobtaining the chrominance signal C2, and a third solid state imagepickup element for obtaining the chrominance signal C3.
 3. The imagepickup apparatus with a horizontal line interpolation function asclaimed in claim 1 or 2, wherein said first vertical phase shift sectionand said second vertical phase shift section are disposed so that thesolid state image pickup elements are located so as to be spatiallyshifted in a vertical direction by the specified pitches p1 and p2. 4.The image pickup apparatus with a horizontal line interpolation functionas claimed in claim 1, wherein said first vertical phase shift sectionand second vertical phase shift section comprise a drive control circuitfor controlling a drive circuit for driving the solid state image pickupelements.
 5. The image pickup apparatus with a horizontal lineinterpolation function as claimed in claim 1, wherein said firstvertical phase shift section and second vertical phase shift sectioncomprise a drive control circuit for controlling a drive circuit fordriving the solid state image pickup elements and said solid state imagepickup elements are located so as to be spatially shifted in a verticaldirection.
 6. The image pickup apparatus with a horizontal lineinterpolation function as claimed in claim 1, further comprising a drivecontrol circuit which performs frame storage drive control, and at thesame time, odd-field reading is effected to some of the plurality ofimage pickup elements while even-field reading is effected to the restof the image pickup elements.
 7. The image pickup apparatus with ahorizontal line interpolation function as claimed in claim 1, furthercomprising a drive control circuit which performs field storage drivecontrol, and at the same time, odd-field reading is effected to some ofthe plurality of image pickup elements while even-field reading iseffected to the rest of the image pickup elements.
 8. The image pickupapparatus with a horizontal line interpolation function as claimed inclaim 1, wherein values of the pitches p1 and p2 satisfy the conditionsthat 0≦p1<1 and 0≦p2<1, respectively.
 9. The image pickup apparatus witha horizontal line interpolation function as claimed in claim 1, whereinthe pitches p1 and p2 are quantities corresponding to a 1/2 line portionof 1 field image.
 10. The image pickup apparatus with a horizontal lineinterpolation function as claimed in claim 1, wherein the pitches p1 andp2 are set as p1=p2=p.
 11. The image pickup apparatus with a horizontalline interpolation function as claimed in claim 1, wherein saidplurality of solid state image pickup elements for obtaining thedifferent three chrominance signals C1, C2, and C3 are a first solidstate image pickup element for obtaining the chrominance signal C1 and asecond solid state image pickup element for obtaining the chrominancesignals C2 and C3, and wherein the vertical phase pitches are p1=p2=p3.12. The image pickup apparatus with a horizontal line interpolationfunction as claimed in claim 11, wherein the value of the pitch p3satisfies a condition of 0≦p3<1.
 13. The image pickup apparatus with ahorizontal line interpolation function as claimed in claim 11 or 12,wherein the pitch p3 is a quantity corresponding to a 1/2 line portionof a video signal to be interlace-scanned.
 14. The image pickupapparatus with a horizontal line interpolation function as claimed inclaim 11, wherein said interpolation circuit performs interpolationprocessing on the chrominance signal C1 with an interpolationcoefficient w (0≦w<1), and performs interpolation processing on thechrominance signals C2 and C3 with interpolation coefficients determineddepending on w and p3.
 15. The image pickup apparatus with a horizontalline interpolation function as claimed in claim 11, wherein the twosolid state image pickup elements are located so as to be spatiallyshifted in a vertical direction by the specified pitch p3.
 16. The imagepickup apparatus with a horizontal line interpolation function asclaimed in claim 1, wherein the different three chrominance signals C1,C2, and C3 are three chrominance signals R, G, and B.
 17. The imagepickup apparatus with a horizontal line interpolation function asclaimed in claim 1, wherein the different three chrominance signals C1,C2, and C3 are three chrominance signals R, G, and B, and wherein C1=G.18. The image pickup apparatus according to claim 1, further comprisinga coefficient generation auxiliary means for generating the specifiedpitches p1 and p2.
 19. The image pickup apparatus according to claim 1,further comprising control means for changing the specified pitch p1 ofthe first vertical phase shift section and the specified pitch p2 of thesecond vertical phase shift section.
 20. The image pickup apparatusaccording to claim 1, wherein the interpolation means further comprisesmeans for separately interpolating the chrominance signals C1, C2 andC3.